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Abstract 

The U.S. Army Digital Training Management System (DTMS) is a web-based system 
designed to create a single point of entry for units to schedule unit training, manage training 
resources, and create schedules and master calendars for training. Currently, the U.S. Army uses 
DTMS to manage unit training and help commanders at each step of the training management 
process from planning and preparing to execute and assessing the training plans. This research 
aims to add intelligent features to DTMS through augmenting it with an intelligent decision 
support system (ITPSS) that utilizes artificial intelligence techniques (case-based reasoning, in 
particular) to determine if training guidance (either annual training guidance or doctrinal 
template) was implemented correctly. The proposed system should also help company 
commanders to refine their unit training plans after reviewing previous similar unit training plans 
recommended or retrieved by the ITPSS. This research demonstrates how case-based reasoning 
could improve the training plan development and approval process in DTMS, and questionnaire 
results support this analysis. It is worth noting that the focus of this research is on the 
applicability and plausibility of the proposed decision system, not on developing an interface 


between DTMS and DSS. 
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1. Introduction 

Large organizations and systems adapt to change (including new technology) or fail. This 
maxim includes the integration of new technology into old systems that will be eventually be 
replaced by new systems and applies to the U.S. Army’s Training Management System (ATMS) 
and Digital Training Management System (DTMS). The Army’s Training Management System 
(ATMS) supports the Unit Training Management (UTM) process to enable and ensure that 
“Training is the primary focus of a unit when not deployed” [1 ]. ATMS includes DTMS, the 
Army Training Network (ATN), and the Combined Arms Training Strategy (CATS). 

DTMS is a part of the “...Army Training Management System (ATMS) (DTMS, Army 
Training Network, CATS) to plan, resource, and manage unit training and access Army standard 
training products” [2], DTMS will, as a part of long-range planning efforts, and future systems 
development, have Artificial Intelligence techniques integrated into the Army Training 
Management System (ATMS) that “Leverage ongoing research in artificial intelligence, adaptive 
learning systems, virtual training and human performance measurement technology” [3]. 

Based on that guidance, it is necessary to understand what DTMS is and where it fits into 
the training management process. DTMS is a “...web-based system is designed to link existing 
systems to create a single point of entry for units to schedule unit training, manage training 
resources, and create schedules and master calendars for training” [4], DTMS was developed by 
nFocus and uses a .net SQL data structure [8]. Currently, the U.S. Army uses the DTMS to 
manage unit training where DTMS is considered a premier training management tool that 
“.. .helps commanders at each step of the training management process from the plan and prepare 
to execute and assess” [5]. Another aspect of DTMS is that it is a program of record and usage at 
every echelon as mandated by Army Regulation, “To assist commanders, DTMS (Web-based) is 
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the system that will be used to track and schedule training and provide summary reports to assist 
in determining individual and collective skill proficiency for assessing unit overall training” [6]. 
Currently, a unit’s (Brigade, Battalion, or Company) long range training calendar is planned in 
DTMS. 

According to the literature reviewed for this research and discussions with ATMS 
personnel at Fort Leavenworth, DTMS is a vital planning tool for the US Army. On the other 
hand, the Deputy Director of the Training Management Directorate at Fort Leavenworth stated 
that DTMS does not currently provide a way for a leader to check subordinate unit planned 
training events in support of a unit’s long-range training schedule or gated training strategy 
against a “perfect plan” [7], Additionally, this is supported by a guidance memorandum outlined 
in June 2014, which supports a lack of intelligent behavior in DTMS due to not being in the scope 
of requested capabilities [8], Case-based reasoning (CBR) seems to be a suitable artificial 
intelligence technique that can allow an automatic check of how close executed plans are to a 
desired perfect plan. 

CBR, generally speaking, is the process of solving new problems by remembering (for a 
human) or retrieving (for a computer) a previous way that worked, called a case, to identify 
similar ways through reasoning that may apply or are similar to the new problem. Humans use 
this type of reasoning daily based on their experience learned from other similar situations 
(cooking, fixing cars, etc.). A key component to using computers to solve a new problem is an 
accurate case base that can be queried based on concrete case attributes [9]. At the highest level 
of generality, a general CBR cycle may be described by the following four processes [10]: 

- Retrieve the most similar case or cases, 

- Reuse the information and knowledge in that case to solve the problem, 
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- Revise the proposed solution, 

- Retain the parts of this experience to be useful for future problem solving. 

Developing a decision support system that uses CBR methodology allows a user or 

commander, in this case, to automatically validate the performance of past plans against future 
plans in DTMS. This feature should support commanders and staff at all echelons. The proposed 
DSS provides a way to compare a recently implemented unit training plan (UTP) from DTMS to 
the perfect plan or “perfect solution” from the Combined Arms Training Strategy (CATS). This 
retrieval provides commanders with a way to compare their plan to other units’similar plans 
(training cycles). An example of this process is a unit that receives a change of mission and has 
to rewrite their training plan for a new area of operations. Based on discussions with ATMS 
personnel, the timing of this study is good due to the ATMS Requirements Control Board (RCB) 
Work Group who met 3-5 MAY 2016 to discuss evolving ATMS functionality and integrate new 
requirements [11], This study does not focus on modifying the current capabilities of DTMS 
directly, but as a proof of concept due to DTMS being a secure “live” system. 

1.1 The Unit Training Management Process 

DTMS supports the Unit Training Management (UTM) process which “...is the process 
commanders, leaders, and staff uses to plan-prepare-execute-assess unit training and leader 
development.” UTM also helps identify the resources needed to conduct effective, performance- 
based training and leader development [12], For this thesis, a discussion on Unit Training Plan 
development and approval are not germane due to the case base consisting of already approved 
unit training plans. 
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The training management process has the following phases as shown in Figure 1, below: 
Phase 1 - Plan - The planning phase includes the planning of a training event far enough out in 
the training cycle to be able to lock in resources needed for successful execution of the training 
event in accordance with the commander’s intent to train on key collective tasks (KCT) [11]. The 
commander is a key part of this process and that explains why the commander is shown as the 
central hub for this process in Figure 1. 



Figure 1. The Operations Process, adopted from [12, p.l 1 (Figure 1-2)] 

The planning phase includes an assessment of the KCTs to determine whether a unit will conduct 

a training event at a crawl, walk or run level based on that unit’s proficiency on the KCTs and 
their assessment. This process helps in the identification of necessary resources for the training 
event. 

Phase 2 - Prepare - The prepare phase for a training event is no different than planning an 
operation. Resource planning and coordination must occur early to ensure that the correct 
resource is available at the right time and right place so that the training event is meaningful and 
effective [12], Different resources have to be coordinated early enough in the training cycle, i.e. 
training area lock-in may occur 60 days out, but ammunition lock-in may occur 90 days out from 
execution. 

Phase 3 - Execute - The execute phase includes rehearsals, and pre-combat checks and 
inspections, time for the execution of the training event, and time to retrain a training event on the 
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Unit Training Plan [12]. By executing a training event that has been properly planned, resourced, 
and coordinated the unit will finish the training event with a higher rating in key collective task 
training proficiency. 

Step 4 - Assess - The assessment phase is an ongoing process that occurs before, during, and 
after training events and training cycles or training phases (see Figure 1, above). The after action 
review (AAR) is a critical aspect of the assess process and an AAR is conducted at the end of 
each training event. Based on this AAR, the unit can determine how well they performed during 
the training event on their key collective tasks, and this evaluation is the basis of the next training 
event plan as a part of the crawl, walk, and run methodology. The assess phase is not a 


standalone phase but is a part of each step of the unit training management or operations process. 



Figure 2. Development of the Unit Training Plan (UTP), adapted from [12, p.10] 

The unit training plan is an output of the UTM process. The UTM is applied to every unit 

in the Army focusing on developing, approving, and executing UTPs. UTMs will be used to 
build the case repository of our decision-based system where one case will represent each UTP. 
The UTPs used for this research are already developed and approved unit training plans (see 
Figure 2). The results from this thesis will act as a proof of concept and will help determine the 
plausibility of adding artificial intelligence into future versions of DTMS through leaders’ 


feedback on the proposed system. 
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1.2 Case-based Reasoning 

The Case-based reasoning cycle consists of four phases: Retrieve, Reuse, Revise, and 
Retain (R4) [13]. “The CBR process requires cases that consist of problem and solution 
description.” [ 16, p. 11. However, “in tackling a real-world problem with an AI solution, it is not 
uncommon to find that a single AI system fails to meet all the requirements for solving the 
problem.” [17, p. 196]. 

Case-based reasoning can utilize numerous types of algorithms to retrieve similar cases 
from the case base to a query case. Examples include clustering algorithms such as k-Means and 
DBSCAN[ 14], case similarity algorithms such as k-nearest neighborhood (k-NN), where k is the 
number of most similar cases[15] or Nearest Neighbor (NN), which are applied in the retrieval 
step of the CBR process. 

1.3 Contribution to the State of the Art 

ATMS consists of three major tools that support the Unit Training Management Process: 
DTMS (Digital Training Management System), CATS (Combined Arms Training Strategy), ATN 
(Army Training Network). Units (Division, Brigade, Battalion, and Company) use the Unit 
Training Management Process to implement ATMS into action. This research focuses on 
enhancing unit training plans in DTMS. 

Although DTMS is a good planning tool, it does not exhibit any intelligence as it cannot 
aid or provide suggestions to commanders during the planning of training events. While the 
DTMS system is a helpful tool, the interjection of artificial intelligence would be beneficial to 
support users by providing an evaluation and retrieval methodology in support of planning. 

ITPSS (Intelligent Training Plan Support System) is a proposed system that uses maneuver 
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company training plans from one Division to develop a case repository. Battalion commanders 
and above could use DTMS as a tool that allows them to see current, proposed plans, and 
automate a previously manual evaluation of those plans during the approval process. An 
intelligent support system called ITPSS is developed for the purpose of this research to augment 
the current capabilities of DTMS so that it can exhibit that desired intelligent behavior. 

1.4 Thesis Outline 

This thesis is organized as follows: The following chapter is the related works section 
covering decision support systems that have used case-based reasoning. Chapter 3 discusses the 
current research and the proposed decision support system, including the system architecture, 
design, and development. Chapter 4 presents the empirical evaluation section and finally, 
Chapter 5 provides a discussion and conclusion of this work. 
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2. Related Works 

The U.S. Army provides guidance on Unit Training Management. The Army does this 
through the implementation of numerous documents: Army Doctrine and Training Publications 
(ADP), Army Doctrine Reference Publications (ADRP), Field Manuals (FM), Pamphlets (PAM), 
and Army Regulations (AR). However, the integration of artificial intelligence techniques is not 
found as a part of current U.S. Army systems or doctrine. 

The Combined Army Center - Training (CAC-T) publishes a quarterly online magazine 
called Army Training Quarterly that is posted on the Army Training Network (ATN) website [18]. 
This magazine addresses current training areas of emphasis that CAC-T has purview over which 
impacts training across the entire force. Examples of topics from the Spring 2016 magazine 
include discussions of the following: 

1. Integrating the Live, Virtual, Constructive - Integrating Architecture (LVC-IA) into 
training at the Joint Multinational Readiness Center (JMRC). 

2. Supporting training in an uncertain environment by sharing best practices of unit’s usage 
of the Integrated Training Environment (ITE) and training aids, devices simulators, and 
simulations (TADSS). 

3. Army Training Management Systems. 

The authors discuss the Army Training Management System and the Unit Training 
Management process in detail to provide leadership examples, best practices, and digital training 
tools to the Army. However, this article only explains or better describe the Unit Training 
Management Process from ADP 7.0 and the Leaders Guide to Unit Training Management [19]. 
The authors discuss the responsibility of leaders to develop their Unit Training Plans to account 
for a crawl-walk-run methodology and that the current digital tools (CATS and DTMS) make this 
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process easier. In [20], the NATO Education and Individual Training Directive establishes the 
guidance for the use of a Training Management System called e-NATO for Education and 
Training but the training management process is not the same as the U.S. Army’s Unit Training 
Management Process, and it does not integrate any artificial intelligent techniques. 

The Training Management Process and the Operations Process are the same process and 
remain a key part of developing unit training plans by doctrine and a crawl-walk-run 
methodology [21]. This formalized structured process allows it to be used for decision support to 
commanders as they “...operationalize their intent and vision for unit-level training and recapture 
the art of training management [19].” By implementing a crawl-walk-run methodology, units 
ensure that training is iterative, progressive, and commander focused [21]. This research focuses 
on the military and civilian integration of decision support systems (DSS) and more specifically 
on DSS that utilize case-based reasoning (CBR) to support the training management online 
resources of the Army Training Management System (ATMS). Case-based reasoning as 
discussed by the author of [22] is “...a model of reasoning that incorporates problem-solving, 
understanding, and learning and integrates all with memory process.”[23, p.5]. CBR applicability 
crosses many domains, but the backbone of CBR is the R4 process (defined in the next paragraph) 
that queries an accurate case base using attributes as the query values for the retrieval phase. 

Research on DSS that uses CBR has been an active and important research area for more than 
the last 20 years, but even though this research area has been applied in medicine, water 
distribution, the courtroom, oil drilling, military decision support simulation systems, and 
strategic decision support it has not been implemented into the U.S. Army’s mission command, 
training management systems, or TADSS. The remainder of this section will focus on a 
discussion on the research of military decision support systems, decision support systems with 
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knowledge management or modeling, and case-based reasoning decision support systems based 
on similar applications. 

2.1 Case-based Decision Support Systems 

“Military actions are complex situations occurring in complex environments. Therefore, the 
decisions taken in this field must be treated in a complex manner.” [23, p. 135]. The complexity 
and the continuously changing environment caused by the force on force operations are not able 
to be replicated in the real world repeatedly enough for a decision support system to be tested 
other than in a modeling and simulation environment. Therefore, it is not surprising that the focus 
is on modeling and simulation exercises for the majority of decision support system applications 
that support commander’s decisions. In [24], the authors implemented a DSS that utilized CBR 
and a Bayesian Belief Network (BBN) for military decision making that focused on critical 
success factors identification to attempt to declutter the information from the battlefield using 
information superiority. The critical success factors, once identified, were the input into the BBN. 
Given the richness and complexity of the military domain made the modeling of scenarios in the 
case base difficult [24, p. 8232]. This demonstrates the challenge with implementing an artificial 
intelligence solution. The authors’ system used the k-NN algorithm for the case based reasoning 
that focused on the distance between cases to attempt to implement the DSS, but the real time 
data acquisition proved to be a challenge when trying to turn uniquely identified features into a 
case for recommending a decision. This DSS was able to identify the critical success factors to 
aid in situational awareness, but the authors believe this system is a better fit for wargaming, not 
practical for real-time decision support [24]. 
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In [25], the authors implemented a case-based decision support system (CBDSS) that focused 
on supporting military command and control by using the standard operating procedure to match 
to the current situation or the new problem and the case base. This system demonstrated through 
testing an increase in subjects who won their games using the CBDSS[25], Nonetheless, this 
system also found that the dynamic ever changing environment proved too much of a challenge 
for their CBDSS to demonstrate consistently reliable decisions. Similarly, the author of [26] 
found this to be true as well. Since most military decision support systems focus on modeling and 
simulation to try to provide real-time decision support, expanding the research was needed. Of 
note, what was not found during research was military case based reasoning implementation in 
support of training management systems. 

When researching decision support systems with knowledge management or modeling that 
use case-based reasoning, an aspect that stands out is the need for domain experts and knowledge 
engineers who provide the expert knowledge for an accurate case base that the CBR tool can 
implement. This expert refinement can be considered knowledge management is an ongoing or 
continuous process [21]. The authors of [21] recommend establishing four knowledge containers 
(Vocabulary, Similarity Measures, Adaptation Knowledge, and Cases) to build a similarity based 
knowledge model for the CBR tool to use. myCBR is “ an open source tool targeting at 
developing customized knowledge models with an emphasis on vocabulary and similarity 
measure development.” [21, p.2], myCBR’s process of formulating a knowledge model is 
directly applicable to this thesis. In [27] the authors utilize case based reasoning with several 
different knowledge intensive similarity measures to improve the efficiency of the retrieval phase 
of the R4 process to allow a CBR system to process very large case bases efficiently. The two 
algorithms discussed were the NN and induction retrieval algorithms, and each algorithm has 
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advantages, but in the end results of this paper, both were inefficient for large case bases. Both 
algorithms can process knowledge intensive similarity measures, and the biggest difference 
between the two is whether or not a case has features that depend on other features. If this is the 
situation, the nearest neighbor (NN) algorithm is the preferred algorithm when features depend on 
features [27]. 

In [28] the authors discuss “...an infrastructure that enables businesses to extract, cleanse, and 
store vast amounts of data.”[28, p. 1 ]. The focus of the paper is knowledge management where 
the goal is to leverage knowledge by converting tacit to explicit knowledge or in other words, 
taking knowledge that is understood by an expert (riding a bicycle as a simple example), and 
converting it to written knowledge like writing the instructions for how to ride the bicycle for the 
tool to utilize. Case-based reasoning utilizes explicit knowledge, and the knowledge worker seeks 
to keep the best cases while removing cases that had higher failure rates. This DSS/IT/AI system 
improves the knowledge warehousing of data to enhance each phase of the knowledge 
management process for the knowledge worker updating the system [28]. This process is in a 
feedback loop with a validation phase of the output thus improving the Al-based data mining 
system. The functional requirements for knowledge warehousing are directly applicable to the 
U.S. Army’s Digital Training Management System and will be discussed further during data 
analysis. In [29], the authors use electronic concept maps for building knowledge models where 
the primary tool being used is CmapTools that incorporates case based reasoning (Discerner) and 
data mining (Extender). The area focused on from this article is the case based reasoning 
application to knowledge modeling. Discerner is utilized for case retrieval when similar 
situations are presented that have been solved before, thus providing an area where CBR is 
applicable based on prior concept maps. Each recommendation must be approved by the user 
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before implementation into the concept map, and the retrieval process was based on indexing 
instead of textual references. The authors were encouraged by the retrieval process results and 
found that by broadening searches, errors were reduced and correct case retrieval was improved 
[29], 

When researching case-based reasoning decision support systems several journal articles 
stood out as applicable to this thesis. First, the authors of [22] address integrating “.. .a case- 
based reasoner, a temporal reasoner, and a scheduling system.”[22, p.196] with the goal of 
improving the planning capability of a real-world system called the “System for Operations Crisis 
Action Planning (SOCAP).” [22, p.196]. This level of integration is similar to the process 
necessary to integrate AI into DTMS once DTMS is linked to other existing systems. “This work 
has also paved the way for a more structured integration of using (1) a common knowledge 
representation language that provides an interlingua for different systems, and (2) a client/server 
interface mechanism that supports location-transparent interprocess communication.”[22, p. 201]. 
The lessons learned through the integration process are applicable. 

Also, looking at CBR usage for planning, the authors of [30] address CBR for marketing plans 
that focus on the retrieval of cases of past marketing plans “...containing strategic planning 
knowledge and experiences.” [30, p. 43]. This system focuses on case retrieval, but the 
adaptation problem is left to the user for evaluation due to the difficulty with adaptation. This 
work uses an XML case representation and a multi-attribute decision making (MADM) retrieval 
method. The DSS discussed is a similar process to what this thesis presents by implementing 
ITPS, but the adaptation of similar cases into the current plan is the user responsibility. The 
strategic marketing planning application from this article based on CBR retrieval that required 
weighting and evaluating the similarity indices appears to have been solved by myCBR which can 
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incorporate similarity measures and attributes that can have attributes that can be weighted [31]. 

In [32], the authors address how case-based reasoning can support strategic enterprise decisions in 
business management where there are complexity and uncertainty, but there are lessons from 
previous experience or analogies present. The authors state that “.. ..CBR does away with the 
classical problem of knowledge acquisition bottleneck in expert systems, as it requires a 
representation of the case and the solution.”[32, p. 4], For this statement to be true, the authors 
propose a 5 step methodology for knowledge representation that when tied to similarity measures 
in CBR (myCBR used) to demonstrate applicability once deciding on the correct case attributes. 

The authors of [32] believe that this approach might be useful but requires further testing. In 
[33], the authors implement a CBR system to support courtroom decisions that proved to be 
efficient and effective. The attributes come from a new vehicle accident, and these attributes are 
the new case inputs into the system for the retrieval phase of the R4 or CBR process. Once 
similar cases are retrieved, a determination as to whether or not the solution is applicable is made, 
and due to this, police officers in England were able to implement the recommended solution 75% 
of the time to reduce courtroom workload from vehicle accidents. Also, this paper addressed the 
time necessary for the police officers to gain the trust of the system, and this took a couple of 
years of use to implement. 

The application of case-based reasoning in the above systems demonstrate how decision 
support systems that use case-based reasoning already support multidisciplinary domains, but the 
systems discussed are limited in their lack of application to military training and their lack of an 
explanation capability for each case. The training domain is a good area for the implementation 
of a case-based reasoning solution due to training cycles repeating every 6 to 18 months based on 
the desired certification level that units must achieve before deploying to a combat training center 
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(CTC). This means that the retrieval of previous similar training cycles or cases would aid units 
in the planning process of home station training (HST) to improve unit training plan development. 
The proposed DSS utilizes case-based reasoning to implement a methodology to support Army 
Training Management Systems, and myCBR includes an explanation feature not present in the 
systems discussed above. The explanation feature of myCBR should be beneficial to future 
versions of DTMS after additional Army systems are linked to DTMS. The explanation feature 
of myCBR will be demonstrated in Chapter 3, and is another aspect that makes this decision 
support system a good fit into DTMS. 
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3. Research Goal and Methods. 

Units develop long-range training plans based off of three primary areas; doctrine, mission 
specific pre-deployment training requirements, and Commander’s Guidance which is usually 
codified in an Annual Training Guidance (ATG). The primary difference between the three 
documents is that the ATG is more mission specific based on the assigned mission of a specific 
unit that is preparing for operations in a specific area of operation. The training cycle for this 
specific mission will include deployment training requirements for that area of operation (e.g. if 
an Armor unit is assigned the mission of training host nation security forces in an African country, 
then the training cycle for that mission may not include a tank gunnery). 

The ATG allows a commander to specify their training guidance and vision including the 
certification level a unit must achieve before deployment to a combat training center. This 
guidance is a key part of the Operations Process and Training Management Process. In many 
training cycles, a unit will find that they are conducting the same training as another similar unit 
that has gone through the same type of training cycle in preparation for a similar type of mission. 
As in the example above, units could benefit from being able to look at other units training plans 
in preparation for training host nation security forces in Africa. In other words, one unit should 
be able to find similar training plans, and utilize them as a basis for the development of their Unit 
Training Plan. Current versions of DTMS do not allow the viewing or querying of Unit Training 
Plans by other units (other than their higher headquarters). If a unit does not have an assigned 
area of operations that changes their training cycle to mission-specific events, then the doctrinal 
template from CATS is the point of departure for developing the Unit Training Plan. This 
research attempts to not only show how a commander or his staff could digitally check on how 
well subordinate units integrated training events from either an ATG or doctrinal template into 
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their Unit Training Plan (First Study), but also show how AI could be integrated into not only the 
retrieval of similar training plans to aid in the development of the Units Training Plan (Second 
Study). 

3.1 Research Goal 

This thesis aims to develop an intelligent decision support system called ITPSS that uses 
case-based reasoning to support training management. The proposed system attempts to use a 
proof of concept to demonstrate areas that need to be improved in a real-world training 
management system in use in the U.S. Army (DTMS). 

3.2 Research Tools 

There are many CBR tools available including myCBR. jCOLIBRI2, eXITCBR, and 
FreeCBR [34], jCOLIBRI2 has the capability to evaluate a case base using three “...strategies: 
Hold Out, Leave One Out, and N-Fold.” [35, p. 134]. jCOLIBRI2 also has the capability to build 
specialized CBR applications that include textual CBR applications, recommender systems, 
knowledge-intensive CBR, data intensive CBR, and distributed CBR with multiple extensions 
that can be integrated into other systems [35], jCOLIBRI2 supports all four phases of the CBR 
cycle [35]. eXITCBR has been utilized in health care applications as an independent tool to 
classify and aid in diagnosis, but the current eXITCBR framework is a JAVA multiplatform tool 
that may also be able to aid in experimentation [36]. FreeCBR is also a JAVA framework but is 
stand-alone and is not as flexible as the previously discussed systems. 

myCBR is an open source CBR tool that can be run on a GUI or with the SDK for 
software development or integration into other applications [31 ]. myCBR also provides 
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explanation support for case-based reasoning and “In object-oriented CBR systems the 
vocabulary consists of numerical, symbolic, plain text, and instance type attributes.”[37, p. 1844], 
The explanation feature of myCBR supports the knowledge manager in several ways. First, it 
provides two kinds of explanations, both forward and backward chaining, but second, it allows 
definitions to be added to each attribute in the case-base that allows nonprofessionals to 
understand why the knowledge engineer chose those attributes [21]. myCBR also provides for 
adaptation rules and this capability is being taken from a beta version to a public release version. 
[31]. 

Among the tools presented above, myCBR stood out in three ways for this DSS. First, 
myCBR was the only CBR tool that included an explanation feature. Second, myCBR is an open 
source CBR tool that allows access to the code so that developers can utilize an SDK (software 
development kit) or graphical user interface (GUI). Third, myCBR can utilize a rule-based 
system for adaptation of retrieved cases. This capability was developed in [31] but is not ready 
for public use yet. 

3.3 ITPSS: Intelligent Training Plan Support System 

The decision support system developed in this research is called ITPSS (Intelligent 
Training Plan Support System). ITPSS uses case-based reasoning as each training plan from 
maneuver companies from DTMS (XML files) seems to be naturally mapped to a case in the case 
repository of the system. The architecture for the ITPSS is demonstrated in Figure 3. 
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Figure 3. Intelligent Training Support System Architecture (ITPSS), adapted from [31, p. 19] 


3.3.1 Building the case base 

The case base consists of 32 cases representing the maneuver companies from the First 
Cavalry Division where each company’s UTP represents one case in the case base (repository). 
The selection of maneuver companies instead of other types of units allows for a better 


demonstration of ITPSS due to focusing on a larger unit base in a Divisional structure giving it 


more applicability (See Figure 4). Battalions highlighted in red were used to develop the case 


base using their company unit training plans. 
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Each case contains 17 attributes that were derived from the Combined Arms Training Strategy 
(CATS) Training Event Matrix due to the Mission Analysis Doctrinal Template being offline. 

Each attribute represents one training event that should be on a training calendar or unit training 
plan as follows: 

1. Unit = the unit name listed as a letter to represent the company the case data 

2. HST months before CTC = the amount of time of training plan data before a unit deployed to a 
combat training center (CTC) 

3. CALFEX = Combined Arms Live Fire Exercise 

4. COMEX = Communications Exercise 

5. DEPEX = Deployment Exercise 

6. FTX = Field Training Exercise 

7. FTX MCTC = Field Training Exercise at the Mission Training Complex 

8. Virtual Gunnery Training = virtual gunnery exercise in simulators 

9. Gunnery Table I-VI, Stabilized (Crew) = live-fire gunnery exercise (tanks firing live rounds) 

10. Gunnery Table I-VI, Unstabilized (Crew) = crew served weapon firing off of the track vehicle 

11. Gunnery Table VII-IX (Section) = section tank gunnery exercise firing live rounds 

12. Gunnery Table X-XII (Platoon) = Platoon tank gunnery exercise firing live rounds 

13. LTX (Platoon) = Lite Tactical Exercise 

14. SGT Time = Sergeant’s Time Training 

15. STX (Platoon) = Situational Training Exercise at the Platoon Level 

16. TEWT = Tactical Exercise Without Troops 

17. TM TNG = Team Training 


Each Company should conduct each training event prior to deploying to a combat training center, 
but variations exist based on certification level, time available, and theater specific training 
requirements. Each case was converted from an excel calendar to a text file from the training 
event comments and then converted to a data excel file for query/vlookup to develop the overall 
case base that was used as the input into ITPSS (See Table 1). 
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Table 1. DSS Case Base used as the base input into the First and Second Studies 
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Figure 5. shows a screen shot for the ITPSS after the case base has been imported to it with the 
blue box highlighting the area the case instances should populate into and the red box highlighting 


the case-base statistics available. 
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Figure 5. Case Base imported to ITPSS 
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3.3.2 Retrieval Algorithm 

ITPSS utilizes K-Nearest Neighbor Retrieval Algorithm (KNN) built in myCBR to 
retrieve multiple cases. The algorithm follows the local-global approach which divides the 
similarity measure into a set of local similarity measures for each attribute, a set of attribute 
weights, and a global similarity measure for calculating the final similarity value. This means, for 
an attribute-value based case representation consisting of n attributes, the similarity between a 
query q and a case c may be calculated as follows: 

n 

Sim(q, c) = £ ' sim Mi » c ,) 

i=1 

Where sirrn and w, denote the local similarity measure and the weight of attribute i, and Sim 
represents the global similarity measure [39, p. 110] 

3.4 System Validation 

In order to validate ITPSS, one case was selected (Case A) and used to query the case base 
to determine if the query case would be retrieved and to what degree of similarity. The results of 
that query can be seen in Figure 6. 

Check of Retrieval Accuracy 



Intelligent Training Plan Support System (ITPSS) Instance Number 


Figure 6. Check of Retrieval Accuracy 
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Figure 6 shows the distance measure as it relates to each instance (case) in the case base. 
ITPSS was verified using the check case retrieval accuracy, and check retrieval consistency tests 
stated in [40, p.35]. Querying ITPSS with a case always returns the query case every time the 
query is run which proves that the system is consistent and accurate. 
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4. Empirical Evaluation 

ITPSS was tested with human participants who were selected based on usage/familiarity, 
or having worked in positions that required the use of the U.S. Army’s Digital Training 
Management System (DTMS) which is part of ATMS. This study aimed to determine if people 
who worked with DTMS would see an added value in integrating ITPSS with DTMS. Using 
experienced individuals who have managed DTMS in leadership positions at the Company and 
Battalion level provides a unique perspective into what this study is trying to accomplish by 
focusing on supporting the usage of DTMS at more senior levels. 

Two studies were conducted to test ITPSS. The first study demonstrates how ITPSS can 
be used to provide similarity measure between all cases (implemented plans) in the case base to 
an expert case (Annual Training Guidance or CATS Doctrinal Template) that is used as the query 
case. The second study shows how ITPSS can help officers design their plans while being able to 
use other plans that are similar enough to their current plan. This way ITPSS supports 
information sharing where units had the capability to view other units training plans when ITPSS 
is integrated with DTMS. The results of the two studies were shared with the human participants 
who have worked with DTMS to get them to evaluate ITPSS and let us know how useful they 
think ITPSS is and if they would recommend integrating it with DTMS. 

4.1. Research Methods 

A questionnaire was designed for this study that provides an overview of the research and 
explains the purpose and goal of the studies (see Appendix A). The questionnaire was created in 
SurveyMonkey for the sake of anonymity. The questionnaire respondents were not tracked by 
name (anonymous). Also, no other respondent data, such as ages, names, or sex was tracked. All 
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participants speak English; either is active duty army officer or is retired officers or senior non¬ 
commissioned officers. All participants participated in this study willfully without any payment 
or coercion for their participation. The results of these questionnaires will be discussed in the 
following subsections. 

4.2. Survey Participants 

The two target audiences for the developed questionnaire are individuals who work in the 
Training Management Division at Fort Leavenworth, KS who manage DTMS and individuals 
who are current or former Battalion Operations Officers or Company Commanders who used or 
managed the usage of DTMS for training management. The survey was sent out to officers who I 
personally know have the background necessary to be included in this survey. Fifty 
questionnaires were sent out, and 25 have been returned. Fifty should be a reasonable size sample 
to allow 99% confidence level and approximately 20% confidence interval [41]. Analysis of the 
questionnaire results and overall questions analysis are discussed below. 

4.3. First Study: Comparison of all cases to an expert case 

This study focuses on how DTMS can benefit from an intelligent support system like 
ITPSS. ITPSS provides a degree of similarity between all the cases (training plans) to the query 
case (expert plan) and presents this on QTB slides. This functionality should allow commander 
or more specifically his staff to determine how well a unit integrated required training events into 
their UTP in comparison to an ATG or a doctrinal template. This should reduce man hours spent 
in preparing Quarterly Training Briefs and should facilitate a commander-to-commander 
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discussion. Most importantly, using ITPSS will allow the use of unified language instead of using 
different ‘language’ by each echelon. 

For this study, the expert case used to query ITPSS is CATS TEM (see Table 3). ITPSS 
determines the degree of similarity for all cases in the case base to the query case and displays the 
four cases with the highest degree of similarity in the results section as shown in Figure 7. 





(a) Expert Case (b) Top Four Retrieved Cases 

Figure 7. Sample retrieved cases for the perfect plan query case 
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Figure 8. Distance measure for the retrieved instances for a query case 
Figure 8 shows that Case 17 (Company T) is the closest case to the expert case with 80% 
similarity. Case 17 is considered the best-matched case due to the highest number of common 
features or attributes with the matching the query case. ITPSS allows the user to see the 
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explanation associated with Case T as shown in Figure 7(b). The explanation feature provides 
additional information about the different events (attributes) and values of the retrieved case. 



Figure 9. Explanation features in ITPSS 

Human participants were mostly satisfied with the results of this study as 84% of the 
respondents agree or strongly agree that ITPSS would be a good addition to DTMS. 64% of the 
respondents believed that evaluating the implemented plans versus the expert plan would be 
helpful for Brigade level personnel, and over half (56%) agreed or strongly agreed with the degree 
of similarity of the top four returned training plans. The opinions of the human participants were 
captured and illustrated in Figure 10. 


Question:lf you were the Brigade Commander receiving the 
QTB, would having the ability to compare your Annual Training 
Guidance or a Doctrinal Template from the Combined Arms 
Training Strategy (CATS) to subordinate units training plans be 
of benefit? 



Question What echelon do you believe would benefit from 
automating the process of evaluating a subordinate unit’s Unit 
Training Plan against either a doctrinal template or higher 
headquarters annual training guidance? 



(a) Would the First study be helpful? (b) Where would the First Study work best? 


Figure 10. First Study Questionnaire analysis 
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4.4. Second Study: Training Plan Retrieval for Comparison 

This study focuses on how ITPSS can aid in the development of a similar Unit’s Training 
Plans. The second study included 32 cases divided into 27 cases left in the case base, and 5 cases 
used as query cases. The different attributes of the five query cases in this study (Case A, Case B, 
Case C, Case D and Case F) are shown below in Table 2. The retrieval results are displayed in 


Table 3. 

Table 2. Query Cases 
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Table 3. Results Comparison Table 


Study #2 Results Listing the Top 4 Similar Cases (Nearest Case / Degree of Similarity / # attributes that are the same / Number undefined attributes) 

Query #1 using 

Case A 

0/0.87/4/2 undefined 

V / 0.84 / 5 / 2 undefined 

AA / 0.84 / 5 / 2 undefined 

BB / 0.84 / 4 / 2 undefined 

Query #2 using 

Case B 

S / 0.87 / 7 / 2 undefined 

H/0.87/6/ 2 undefined 

R / 0.84 / 6 / 2 undefined 

V / 0.84 / 6 / 2 undefined 

Query #3 using 

CaseC 

V/0.87/5/2 undefined 

BB/0.87/4/2 undefined 

1 / 0.87 / 3 / 2 undefined 

S / 0.84 / 5 / 2 undefined 

Query #4 using 

Case D 

S/0.87/6/2 undefined 

H / 0.87 / 5 / 2 undefined 

R / 0.84 / 5 / 2 undefined 

V/0.84/5/2 undefined 

Query #5 using 

CaseF 

GG / 0.91 / 5 / 2 undefined 

H / 0.91 / 5 / 2 undefined 

R / 0.87 / 5 / 2 undefined 

U/0.87/5/2 undefined 


Table 3. shows that the top retrieved case is Case A with 87% similarity to the current plan under 


construction. The advantage of viewing other unit’s training plans by a query process is not a 
current capability of DTMS. However, by allowing units to retrieve and view similar UTPs, 
DTMS would be sharing knowledge across formations to provide Company Commanders the 
ability to find units who conducted similar training cycles to view their UTPs and validate the 
current plan against another solution to the same problem. This allows information sharing across 
formations to improve Company Commander’s UTP development for specific mission focused 
training plans (as in the case when a unit is assigned the mission of training host nation security 
forces in Africa, details mentioned earlier in the thesis in the introduction to Chapter 3). 

















[Dugger] 36 


Moreover, if a Commander knew which unit completed similar training cycles for similar 
missions, he/she would be able to contact those units to discuss lessons learned from the units 
returned (previous training events) for a particular query (perfect plan). A commander may also 
decide that a retrieved case should be adapted and used as his unit training plan. It is worth 
noting that adaptation is not a current capability of ITPSS. 

The second study showed that 80% of the human participants either view this capability in 
ITPSS as helpful/very helpful if it becomes part of DTMS. 64% of the respondents agree with the 
similarity measure (degree of similarity) between the retrieved cases and the current plan under 
construction (query case) (See Appendix A). 


Question: How helpful would the ability to look at other 
units' similar training plans be to aid in the finalization of 
your unit training plan? 



Neutral. 


Somewhat 
Helpful. 


Helpful. 4 0% 


■ Very Helpful 

■ Helpful 
oNeutral 

□Somewhat Helpful 
■Not Helpful 



(a) Participants opinions on the usefulness of ITPSS (b) Recommendations on ITPSS usage by echelon 


Figure 11. Results from human participants 


Figure 11 b shows that 68% of respondents believed that the ability to retrieve similar units 
training plans should be resident at either the Company or Battalion level (second study). Only 
one respondent recommended this functionality at the Division level, and this may be due to the 


fact that Division level campaign plans are very specific to a theatre of operations. This makes 


them less likely to glean from adjacent unit training plans or different campaign plans. Also, 


respondents would be less likely to select the Division level for this capability due to training 
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doctrine where headquarters only certify two levels down (Divisions train Battalions, Brigades 


train Companies, etc.). 


Question: If you could re-arrange the plans suggested by 
4 the system, what would your ranking be? 
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Figure 12. Second Study case retrieval ranking chart 

The human participants were asked if they agree with the output provided by ITPSS in the 
sense of similarity measure agreement. Figure 12 shows the number of participants agrees and 
disagree with ITPSS’s output and their suggested output. This question had the biggest variance 
(44%) among all the questions. The majority of the respondents disagree with the ratings from 
the ITPSS. A possible reason is that the plan duration was not specified in the query case, so each 
respondent evaluated the retrieved cases based on their operational and/or training experience). 
Case R had three months to train and case U had five months to train for home station training 
time before the unit went to a combat training center. This is reasonable, and experienced officers 
would have picked the instance with five months over the instance with three months. 
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5. Discussion and Conclusion 

5.1 Discussion 

The integration of AI techniques into future systems is already a part of planning and 
development guidance, and it is only a matter of time before AI techniques are implemented into 
many if not all Army Mission Command and home station training (HST) support systems. 

DTMS (Digital Training Management System) is no exception. This research proposed an ITPSS, 
an intelligent decision support system that can add intelligent capabilities to DTMS. ITPSS offers 
the ability to query a repository of training plans (case base) using case-based reasoning to either 
retrieve similar cases to aid the construction of a plan or to provide a degree of similarity to 
determine how good a training plan is compared to an expert plan. This capability would help 
commanders at each step of the training management process from the plan and prepare to 
execute and assess a unit training plan. 

ITPSS is implemented outside DTMS due to some restrictions and limitations such as the 
researchers not having access to the code for DTMS due to DTMS being a real world, live system. 
Before integrating ITPSS into DTMS, three necessary requirements need to be implemented in 
DTMS. The first requirement is that DTMS allows units to query other units training schedules. 
DTMS was designed around the commander owning his or her training schedule and only through 
changing permissions can another unit view their unit training plan. Based on survey results, this 
capability should be looked at for future versions of DTMS. 

The second requirement is to link other Army systems to DTMS and unify the naming 
conventions used to name each training event (every unit abbreviates the training event names 
differently) so all systems can talk to each other. And finally, the third requirement is to allow 
DTMS to output a CSV file that can be used as an input to ITPSS. If ITPSS cannot be integrated 
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directly in DTMS due to timing or Certificate of Networthiness then, at least, the technique used 
in this research can be implemented directly by units. Results from this research showed that 
expert users of DTMS (84% and 80%) indicated that the capabilities presented by the ITPSS 
would be a good addition to DTMS. Also, 76% of them indicated that they would be either likely 
or very likely recommend ITPSS be integrated with DTMS. 

An additional area that could aid in the development of future versions is that Home 
Station Training Support systems that manage land and ranges, ammunition, and money would be 
linked to DTMS. Once the three requirements mentioned above are implemented and Home 
Station Training Support systems are linked to DTMS additional work will be needed to modify 
the interface of DTMS by adding tabs for the land and range system, the ammunition system, and 
the money system. Appendix 3 provides a recommended interface in support of this effort. 

Not only does this research support future system development guidance, it demonstrates 
that AI integration does not have to wait for the future system to be developed, but can be added 
to existing systems in use today. 

5.2 Conclusion 

Current planning guidance for future Army system development is to leverage artificial 
intelligence research into future system development. ITPSS is an intelligent system that uses 
case-based reasoning technique to support the unit training management process, thereby 
improving the unit training plan development and approval phases. Integrating ITPSS with 
DTMS should allow DTMS to exhibit the desired intelligent behavior sooner than 2030. 

The research results show the usefulness of comparing all previously implemented plans 
in the case base against an expert case and how the human evaluators highly recommend adding 
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this functionality to DTMS to provide an automated evaluation tool for commanders and staff. 

The results also show how the human participants highly recommend the ability to retrieve 
similar cases or training plans to support commanders at all echelons who are in the unit training 
plan development and approval process. 

ITPSS uses myCBR which is open source, and that makes it a cost effective solution for 
the DTMS developers. Integrating ITPSS into DTMS will allow it to transform to I-DTMS that is 
DTMS with intelligent capabilities. Accordingly, this would improve the planning capabilities of 
DTMS and improve support to commanders and staff of all units in the Army. 

5.3 Future Work 

Future work includes adding adaptation rules to ITPSS which will help refine the solution 
of retrieved cases to fit the query case. Additional work needs to be done on the explanation 
feature in myCBR to allow the explanation feature to be part of the system output. Lastly, 
investigate the best way(s) to integrate ITPSS with DTMS. A recommended interface can be seen 
in Appendix C. 
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Appendix A. Second Study Query Results 
Second Study, Query #1 with Case A 
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Second Study, Query #2 with Case B 

The second query was conducted with Case B’s “YES” values of CALFEX, FTX MCTC, 
Virtual Gunnery Training, Gunnery Table I-VI, Stabilized (Crew),Gunnery Table VII-IX 
(Section),Gunnery Table X-XII (Platoon), Platoon STX, and TM TNG. 
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Second Study, Query #3 with Case C 

Gunnery Training, Gunnery Table I-VI, Stabilized (Crew),Gunnery Table VII-IX 


(Section),Gunnery Table X-XII (Platoon), and Platoon STX. The results of this query are below. 
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Screenshot of Second Study, query#3 with Case C as the query case results 





































































[Dugger] 48 


Second Study Query #4 with Case D 

The fourth query was conducted with Case D’s “YES” values of CALFEX, FTX MCTC, 
Virtual Gunnery Training, Gunnery Table I-VI, Stabilized (Crew),Gunnery Table VII-IX 
(Section) ,and Gunnery Table X-XII (Platoon). The results of this query are below. 


myCBR 


File Model Case Base 


i □ Projects 


f & & & X 


i £ Study2Revised 
d 0 Sustainable Readiness 
® CALFEX 
® COMEX 
® DEPEX 
® FTX 
® FTX MCTC 

® Gunnery Table I to VI Stabilized Crew 
® Gunnery Table I to VI Unstabilized Crew 
® Gunnery Table VII to IX Section 
® Gunnery Table X to XII Platoon 
® HST months before CTC 
® Platoon LTX 
® Platoon STX 
® SGTTime 
® TEWT 
® TM TNG 
® Unit 


4 Retrieval: Sustainable Readiness 

Retrieval 

Case base: Study2CaseBase v 
Query 

CALFEX 
COMEX 
DEPEX 
FTX 


(Quid 


Modeling £3 Case Bases fi & 



NO 


YES 


FTX MCTC YES 

Gunnery Table I to VI Stabilized Crew YE ^ 

Gunnery Table I to VI Unstabilized Crew 
Gunnery Table VII to IX Section 
Gunnery Table X to XII Platoon 
HST months before CTC 
Platoon LTX 
Platoon STX 
SGT Time 
TEWT 


NO 




TM TNG 
Unit 

Virtual Gunnery Training 


NO 

.unknown. 

(yes 


Change 

Special Value: none 
Change 

Special Value: none 
Change 

Special Value: none 
Change 

Special Value: none 
Change 

Special Value none 
Change 

Special Value none 
Change 

Special Value none 
Change 

Special Value none 
Change 

Special Value none 
Special Value unknown 
Change 

Special Value none 
Change 

Special Value none 
Change 

Special Value none 
Change 

Special Value none 
Change 

Special Value none 
Change 

Special Value unknown 
Change 

Special Value none 


Sustainable Readinessl 1 - 0.87 
Sustainable Readinessl - 0.87 
Sustainable ReadinesslO - 0.84 
Sustainable Readiness14 - 0.84 
Sustainable Readiness12 - 0.84 
Sustainable Readinessl3 - 0.84 
Sustainable Readiness20 - 0.84 
Sustainable Readiness2 - 0.84 
Sustainable Readiness25 - 0.8 
Sustainable Readiness7 - 0.8 
Sustainable ReadinessS - 0.8 
Sustainable Readinessl 5 - 0.77 
Sustainable Readiness19 - 0.77 
Sustainable Readinessl 6 - 0.77 
Sustainable Readiness21 - 0.77 
Sustainable Readiness22 - 0.77 
Sustainable ReadinessS - 0.77 
Sustainable Readiness26 - 0.77 
Sustainable ReadinessS - 0.77 
Sustainable Readiness18 - 0.73 
Sustainable Readiness17 - 0.73 
Sustainable Readiness23 - 0.73 
Sustainable Readiness24 - 0.73 
Sustainable Readiness6 - 0.73 
Sustainable ReadinessS - 0.73 
Sustainable ReadinessO - 0.69 
Sustainable Readiness4 - 0.69 


Similarity 

CALFEX 

COMEX 

DEPEX 

FTX 

FTX MCTC 
Gunnery Table... 
Gunnery Table... 
Gunnery Table... 
Gunnery Table... 
HST months b... 
Platoon LTX 
Platoon STX 
SGTTime 
TEWT 
TM TNG 
Unit 

Virtual Gunner... 


Sustainable Re... Sustainable Re... Sustainable Re.. 
0.87 0.87 0.84 


Sustainable Re... 
0.84 


YES 

YES 


YES 


NO 

NO 

NO 


NO 


NO 

NO 

NO 


NO 


NO 

NO 

NO 


NO 


NO 

YES 

YES 


YES 


YES 

YES 

YES 


YES 


YES 

NO 

NO 

mum 

NO 


NO 

YES 

NO 


NO 


YES 

YES 

HBHUHH 


YES 


NO 

YES 





Screenshot of Second Study, query#4 with Case D as the query case results 
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Second Study, Query #5 with Case E 


File Model Case Base Help 


myCBR : 


j E Modeling Case Bases Bf 1 Sb 


) Platoon LTX 
i) Platoon STX 
) SGT Time 
) TEWT 
) TM TNG 
) Unit 

) Virtual Gunnery Training 


Save results 


Sustainable Re... Sustainable Re... Sustainable Re... 
0.91 0.87 0.87 

II 


□ Case Bases " Instances 
0 Study2CaseBase 



H Projects , <3f X ° □ 

^ Retrieval: Sustainable Readiness >3 ; 



' - = 

□ 

a £ Study2Revised 

a 0 Sustainable Readiness 

Retrieval 




- 

© CALFEX 
© COMEX 
© DEPEX 

Case base Study2CaseBase v 

Query 





© FTX 
© FTX Mac 

© Gunnery Table 1 to VI Stabilized Crew 
© Gunnery Table 1 to VI Unstabilized Crew 
© Gunnery Table VII to IX Section 

Gunnery Teble X to XII Pletoon 

CALFEX 

YES 

Change 

Sustainable Readiness25 - 0.91 




Special Value: none 

Sustainable Readinessl - 0.91 


COMEX 

NO 

Change 

Sustainable ReadinesslO - 0.87 




Special Value none 

Sustainable Readinessl 3 - 0.87 


DEPEX 

NO 

Change 

Sustainable Readiness22 - 0.87 


© HST months before CTC 
© Platoon LTX 
© Platoon STX 
© SGT Time 
© TEWT 
© TM TNG 
© Unit 

© Virtual Gunnery Training 



Special Value none 

Sustainable Readiness20 - 0.87 


FTX 

[no 

Change 

Sustainable Readiness9 - 0.87 




Special Value none 

Sustainable Readinessl 1 - 0.84 


FTX MCTC 

YES 

Change 

Sustainable Readinessl 8 - 0.84 




Special Value none 

Sustainable Readiness? - 0.84 


Gunnery Table 1 to VI Stabilized Crew 

YES 

Change 

Sustainable Readiness23 - 0.84 



Special Value none 

Sustainable Readiness24 - 0.84 


Gunnery Table 1 to VI Unstabilized Crew 

NO 

Change 

Sustainable Readiness5 - 0.84 




Special Value none 

Sustainable Readiness14 - 0.8 



Gunnery Table VII to IX Section 

NO 

Change 

Sustainable Readinessl 2 - 0.8 


C Case Bases □ Instances [£" X D □ 


Special Value none 

Sustainable Readiness19 - 0.8 


0 Study2CaseBase 

Gunnery Table X to XII Platoon 

NO 

Change 

Sustainable Readiness21 - 0.8 



Special Value none 

Sustainable Readiness26 - 0.8 

Sustainable Readiness8 - 0.8 



HST months before CTC 

.unknown. 

Special Value unknown 



Platoon LTX 

NO 

Change 

Sustainable Readiness2 - 0.8 





Special Value none 

Sustainable ReadinessO - 0.8 



Platoon STX 

YES 

Change 

Sustainable Readinessl 7 - 0.77 





Special Value none 

Sustainable Readmess6 - 0.77 



SGT Time 

NO 

Change 

Sustainable Readiness3 - 0.77 





Special Value none 

Sustainable Readinessl 5 - 0.73 



TEWT 

NO 

Change 

Sustainable Readinessl 6 - 0.73 





Special Value none 

Sustainable Readiness4 - 0.73 


© FTX MCTC 

TM TNG 

NO 

Change 


© Gunnery Table 1 to VI Stabilized Crew 



Special Value none 



© Gunnery Table 1 to VI Unstabilized Crew 

Unit 

.unknown. 

Change 



© Gunnery Table VII to IX Section 



Special Value unknown 



© Gunnery Table X to XII Platoon 

Virtual Gunnery Training 

| YES 

: Change 



© HST months before CTC 


Special Value none 




Screenshot of Second Study, query#5 with Case F as the query case results 
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Appendix B. Decision Support System Questionnaire 


Research and Thesis Questionnaire 


Date 


Study explanation and goal: 





myCBR 


Please let us know what is your degree of familiarity using DTMS? 


OSubject Matter Expert Familiar OModerately Familiar 


OSoroewhat Familiar 0 Un ^a m <l , a r 


This research aims to develop a decision support system (DSS) that can aid decision makers in the US Army 
and that can be integrated with the U.S. Army Digital Training Management System (DTMS) in a way that allow DTMS 
to exhibit intelligent behavior. The purpose of this study is two folds: First, determine if a training guidance, either 
annual training guidance or doctrinal template was implemented correctly (Scenario 1). Second, determine if the 
system can aid/guide company commanders to refine their unit training plans after reviewing previous similar unit 
training plans recommended by the DSS (Scenario 2). This research presents a new way to utilize unit training plans 
that is not currently available in DTMS at the time of this study. 


In the meantime, all maneuver companies training plans in a Division are saved in the system as an excel 
sheet with the plan ID or company name in the first column and all the events displayed in the rest of the columns 
(See below) 



Aside note: the focus of this research is on the performance of the proposed decision support system and not on develops an interface 
between DTMS and the DSS. 


Now please read the scenarios below carefully and answer the questions. 

First Scenario: 


You are an Armor Company Commander in 1* Cavalry Division and your unit has been notified they are deploying to 
an area in AFRICOM in twelve months. Based on this notification and training guidance from AFRICOM, you have to 
rewrite your unit training plan to accomplish all required training, which includes a train-up at home station and 
deploying to a Combat Training Center (CTQ. The level of certification your Company must reach prior to the CTC is 
Platoon Table XII, and your Brigade Commander wants to conduct a CALFEX. In this scenario, DTMS has the 
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Pability to allow you to view other units training plans after you feed the system with which training events you 
an to conduct. You can see your plan on the left side and the suggested plans by the system on the right side of the 
jure below. 


jery Input (Your Plan) 

Jtput 



micx 









Search Results or 


0.84 represents there is 84 percent 
simiarfcy between this plan and your 
plan 




Do you agree with the which plans were retrieved (87% & 84% similar), based on the training events entered into 
the query from your unit training plan? 

OStrongly Agree 0 Ne utral 0 Disa g ree OStrongly Disagree 


Do you agree with the similarity of 87% for plan M and 84% for plan R as the similarity of the retrieved plans 
versus your query? 

O Strongly Agree Ogree ONeutral O Disagree O^trongly Disagree 

If you could re-arrange the plans suggested by the system, what would your ranking be? 

1*_ 2 nd _ 3 rd _ 4*_ 

How helpful would the ability to look at other units' similar training plans be to aid in the finalization of your unit 
training plan? 

OveryHelp O Neutral Q Somewhat Helpful O N0tHe lpful 
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What echelon would benefit from this type of search / query? 
ODMsion Q^'g^e O Batta,i0n Company 


Second Scenario: 

You are Company "T"s commander, and you run it has finished updatingyour unittraining plan. You now have an 
approved unit training plan for the home station train-up for the CTC rotation and later AFRICOM deployment. 
However, you must prepare for a QTB for your Brigade Commander who is very concerned that all required training 
events were planned into every unit's training plans, m this scenario, DTMS has an automated way to use the query 
to score how well a subordinate unit integrated training events into their unit training plans. Three of the top four 
scores are in your Battalion (Company R, S, and T) whose results are listed below. 



Putting yourself in the commander's shoes, do you agree with the similarity ranking of the units in your Battalion 
(Company R, S, and T) based on the training events that are a part of your unit training plan that were compared 
to Best case or Doctrinal Training plan? 

O Strongly Agree 0 A g ree O Neutral O asa g r ee O^o^Y Disagree 

If you were the Brigade Commander receiving the QTB, would having the ability to compare your Annual Training 
Guidance or a Doctrinal Template from the Combined Arms Training Strategy (CATS) to subordinate units training 
plans be of benefit? 


OVery Beneficial QBeneficial QNeutral Detrimental Qs^^gty Detrimental 
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What echelon do you believe would benefit from automating the process of evaluating a subordinate unit's Unit 
Training Plan against either a doctrinal template or higher headquarters annual training guidance? 

O Division OBrigade O Batta,l0n 


How likely would you be to recommend the proposed system to be integrated with DTMS? 
Qvery Likely 0 Like, Y 0 Neutral 0 Somewhat Likely QNot Likely 
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Appendix C. DTMS Resource Tab Recommended Additions 


Digital Training Management System (Livel) 


MSTl TKiWrajr Task; SokterMaruger CATS Calendar Coune Manager 


Event Schedule 


Unit 


Uxabora OcckUst OSU Resource* 


I "| owe* 



1. Recommend adding areas for Funds allocation. 
Ammunition allocation. Land scheduling and 
Supplies Ordered to the existing event resource 
tracking section for each training event. 


1 Nity Terms mt tl» l> »a «t » « »t Wi Sk u k w 


Digital Training Management System (Livel) 


Twang UETl TadcManager Tasks Sctem Manages CATS Calendar 


Event Schedule 


Unit 


Reporting Http Qa>dMs 


cfc Urt cai kiants Rest*' Task Completion 


Training event 


At CAT'S DODO CA7> Ncn-UN* 

fv*. oc 0*f0* TADtt VMn 

h ~ w "“ s 1 ZIL 1 


discrepancies 


Scheduled Land / Ranees for this Training Event: 





Maneuver Area 

Training Areas 

Ranges Firing Points 

1 ??? 

] 








MA Bravo (19-23FEB16) 

TA 34,35,36,37 (19-23FEB16) 

Range 19 (22FEB16) 




Closest Similar Training Event Historical Land / Ranee Usage Tvpe of Training Event: 




-1 

Maneuver Area 

Training Areas 

Ranges Firing Points 




MA Alpha and Bravo (4 days) 

TA 1,2,3,4,34,35,36,37 (4 days) 

Range 19 (1 day) 



~~1 


| Digital Training Management System (Livel) 


| Unit 

►low Tranng METl 'liUmjr Taste SoUciUmgn CATS Calendar ComUrngn Reporting HHp QwtMs Um 

BOH 



Event Schedule 


Unit 


LEGEND: 


Order Status 


Class II Ordered (populated from PBUSE) = 

Class III Bulk ordered (Populated from SAMS-E) = 


AS CATSOOCXt CATSSor'TI^ (vti Vftxte 

| Fun<h/GfUS | 

Und/ 


Budeeted Funds for this Trainine E \h±b*T 



_ 


Class 1 Class ll jpadfT II Cl^erfV 

Class V 

Class VIII 

Class IX 

GCC 

(Food) (Supplies) (POL) ^^Construction Materials) 

(Ammo) 

(Medical Supplies) 

(Maintenance) 

(Porta Johns, etc) 

$0.00 $0.00 ^ $0.00 ^ $0.00 

$0.00 

$0.00 

$0.00 

$0.00 


Closest Similar Training Event Historical Cost for this Type of Training Event: 




Class 1 Class II Class III Class IV 

Class V 

Class VIII 

Class IX 

GCC 

(Food) (Supplies) (POL) (Construction Materials) 

(Ammo) 

(Medical Supplies) 

(Maintenance) 

(Porta Johns, etc) t,pU "* t, ° r 

| $1000.00 $500.00 $4000.00 $200.00 

$1500.00 

$500.00 

$225000.00 

$1200.00 


















































































